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7':*-:?v'x^ h^&fFfiEf sublet*. HUsB^a<^>P<x-f 
BulBIS^v/U^P*^^^:^^^-^'!^ h^-fp^r 5 

3 ] MiB#$SS1t6'«/ T^-f Z^t ^i^i H © 

7'^:/->'x^7 h^&f^fig-r sublet*, gjtB«ii«Dp<7=^-f 

^aito. ^T<^B9fB^a(^;^x'f x^^ KDMIB 
egSg6^B3iB?*jlSi5HJ-XrtT'^nf#. SaiBB3IB«»(^^ 
7='-<7':t:?'>>x^' f«<7)f*'T'g:^o?i^S*l3iB^'&^:?' 
v»x h <0 S^Si: HuiB«ili5ffl<DTI?«B9fB^^ 

iB^ii@ffin<7)<&(0A^£n(^. e3§B^)Sisra(DS:^fi% 

v>x h<7)g:A:#tf fi« 0 i: LTffi^^/Uf'/ f^-f 
[000 1] 



[0002] 

S14«Jt-:?«»<Dp«x-f7'7t-7>'v>'x'> h^!!i3^^fc-ti:T 
-• p<7)f^B°p (m^^/U5';><x-i'7'^h:^->"x^ h) ^f^fig 

it. ■(5I6*^<3?mT'P<7^i'7':^:?v*'x'? hP^<^)I^^H^^ 

T^y''Jx<7 htiix *i^v;u^^x'f7':d-:^i>x^7 h 

[0 0 0 3] <St^<D^mi/7.=rl^Tit. S^x-fT'^t:?' 

^F^fft±lt«6ff«6V5r^-7':?f:?'v»x-? f-^M'^T?7< 

[0 0 0 4] CfD^T^^l/Tli— iKWlCv 0 2 (a) JJd: 
t?ll2 (b) (C5^f J:^tJ:fil:^[fiimF5K6U:7a-:?ft^8Jf 

h^fjy^TTl^So p<x-i'7':t:?'yx'^ h 
St- ^-ODBtF^SlcJtlJyrSsAcS^OJEimirLTSS^? 

7j-:?v'x'> hF^CJluW^JiSf *c:<t:icS5o 
[0 0 0 5] co^f-<A5'r>^7=^;ujK5Cic<i:s^7^'f 

1Sa<D;'<T'f-7'^^v'x>> htDPlMSra^lis B$Fai4 
±0|5l Cffi^lcBBB* ns C <!: ic «!: -P TIBkE* nSo 
[0 0 0 6] 

[^W«<<»i*LJ:-5<!:-r*aiS] \^ti'l,ts.3^'^<r)l5 
^(cj5t>T«^ ^®^?>U^;<x^'7'7t-::''v>x^7 h^^fc 
J:t;ilfIE-r«PSlc, J.XT<DF.gSjS*'<??aL/co 

[0 0 0 7] (1) IE«Cl4*'<^t^ 

-f 7':t:?'v''x-5' h^^t§3!^^^^•^*<t^^^:^ — :^Ofi*lc 

[0 0 0 81 m«\ ^^'y-^tBCM^m^-^t)^ 
^/7=''f7':!i-:?'5>x^' h^^f^figr^ii^^^^So -r^ 

■<y'E9Mzi^^m$ki/7.=rL.T'\i.. 03 (a) IC^f 
J:-7lc, h^y-J^ 1 lc->->K h^y^2lcBGM^ 

B G Ma3a^A(''-S5f-r * J: lcSlf^«*':iBS-r «<. 
[0 0 0 9] Z.<OWi-^-^l\/'^}^Tr'<T:tyjx<7\'\,Z.ti 

'^^oyfj^T^t. 1213 (b) <^d;7(CBGM 
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«*-a*-e-s/i:4&tctt, E3 (c) (D^vimf^m^ 
[0 0 10] (2) miEtmm 
X -1- 7' * 7" X •J' h F^(D <tr(Dfas«<*^ ic i^wr -^^^-t 

[0 0 1 1] IJlJjllSx E4 (a) lCni-$n^m^^H''^ 
^T^f 7'7»-:^v»'x-^ ^<^*^ V i d e o4a)B35g<tBtlCV 
i d e o Itl^ih^^J. Vide o 4a)!!«7i:lRl^lCV i 
d e o 3tmTr^tl^^—m3i-<Dm-^y(7'^T:f:f'J 

x<7h (cn^a^^x-f 7'7i-:^v'x<7 h 1 tr*) 

VideoltAudiol «^m3f-&t>-l±/ct<DT' 
a&So :^f*t LTtts VideoliAudiol 
^icMJ&Lx V i d e o 1 6^i^7SLJ#6 < LT*^6a 
^/x-f7':t^v' x-57 h 1 6^MSfe-r*c ^■LT«-&;^x 
-f T' x-? h 1 <0f)^7tAu d i o 1 0^7 1"^—^ 

[0 0 1 21 ^:<7)S^^;^5^:><x'r7'?^:/-:>■x'? KD— 

SP*<^IEt"51i^^#jl*o fi<Jx.tf A u d i o 1 :g 
l^trotcSMfSt-rSo +f'fX©^Mlcflil\ Aud 
i o 1 t<Dmmm%ic^-^tzV \ d e o 4©Mii&ttS^ 
^ML^ftttniSSef, *61CV i d e o 4i(DlRl«BM 
^iC^^y-cV i d e o 2j5<*;t;v i d e o 3lC-Pt^T 

«Ji35:^'&"7;U^-^x-r7'7}-:?v*'x':7 h*f^fi£/fgiE<D 
[0013] ( 3 ) giJp°p <!: L TCDSf 'Jffl1t««et> 

gP'feL<«^gp«s ■fte<D«^^;i/^p<x-<7'?r:^'v^x-j7 
^■7;u^p<x-f 7':t:^v''x'j7 ^•^cs!a*^!a,t>pgl^c^o^63«t 

[00 1 4] flW?H#^ E15 (a) (*> BWStJStlSW^— 



[0 0 15] c:0J:5lc^ ?Jt3l5<D«^?/U^^x-f7'7»- 

[0 0 1 61 *f«W(Dea*it*> ±fB<DP<as^^atu fj 

^^^/Uf^yx-<7':d-:/v'x'? h*f6<D«^^;l/^J><x 

So 

[00 17] 

[0 0 18] *gsmiro^R9», tt»«i±Tex;d-^^ 

[0 0 19] *lim2CD«^«. B0iBffaiS146'«/x-f 

B5fB«a<D^x-i'7';t:/i?x-5' hrog^:gro*aJ:y^t^ 
iS^v;u^;><x-f T'^l- x-? h^f^figf SJ^^ia*. 

BufB^»©^x-f 7':t:^v'x-57 h«tJfOg:^Sfl*iacOJt 
v'x^ h**ii'hLByiBffl^v;U^><x^-7':j-y>>i^ h 
[0 0 2 01 *Bim3<7)^R«ttv HiJIBffl«im*Vx^' 

B5iB«S$<Dp<xf 7':t:'i>'x-? hOg^:l©?Q«fey:lt^ 
[0 02 11 *gim4a)S6^tts HuiB^ffiJlttfeVx-r 

^■rSISltx B9i3«ifc<DP<x'r7':t^>>x'^ h<Dg*© 

y-J:L<p KDBuf3g?!SS«<BiJfB*^a®HWrtT'ifent#. 
B!}IBH4fB«SiC©P<X'r7':l-:''v'x-? hCD't'T'g^WiJSS 
S*B3f3«^:j-^v'x'5' KDgJSStLs BuflB^iitSH 
WTW^h3IB«^^:/>>x<7 h©a:A:*g?^atU h?IB 
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[0 0 2 2] 
10 0 2 31 

[0 0 2 4] 06lC^-r<fc-5lC. S^7=^-f T'^ x<7 
hli. Tnatural (S^g) J . Ts h r i n k 
m±ffi3fM.) J ^ r s t r e t c h (g^#tf «) J 

natural /\°^/-^'Ta5'+i«***i:^^<. $ 
fc. ;.<x-f 7':d-:? v> x'? h <^)S:^#tf s t r e t c 

7=^-i'7'*;?v>"i'5' h<^**3-l*> natural + 
s t r e t c hTa^n^^SS-lc^So 
[0 0 2 5] SttrH^tm^lC. yiT'^T^ZTv^x^^b 
<Dm±tiS3i-Mli s h r i n k/\°^p<— ^T'SS'tl^tD 

i't*^ natural - shrink TStftlS 

[0 0 2 6] srcffi-^^yi/^^x-f z:!r-:/v'x^7 h^^fii 

l-F^CD!iS-&a5»a)C<!:^T emp o r a I G I u e «5: 
[0027] Tempora IGI u eWSSAlcJ:*;, 

X'f7';!i-:^i>'x'> g?sic#seT'^« 
[0 0 2 8] ±iBa)«fc-5ic#*SJSit^-^^en/'c;^x^' 
■yV:!.'? h«fffiK■r%fci&0Dj*f^*^73«^;i-:^>>■x'? 



[0 0 2 9] (a) ^P^Wlcaigllcyx-i'T'^i-Z/i^x'? 

h«it^-r««^:d-:?'v^x^ h 

^rSffl-&^:/v*x-i^ StartEndBox 
tfl^'Vo StartEndBox li. ffl»<D^ T*:?!" 

^;><x-<7'pf-^'i>x^7 h*f^^r5«ssf^*?T3«^:i-:?" 

i^'x-? hT-S*„ StartEndB o x:t:?v''x<7 h 
■To 

[0 0 3 0] 117X^1*, (natural, shrin 
k, stretch) ©S#*S/\°^y— S^SMl 0, 
2, 7) (Dyx'f7'*:?'v'x<7 hAi. (15, 10, 
1) £D^x-<7':t:?"v>'x^' hB<^2o:&BtF^«il::)a!ISSl!: 

x-^h^StartEndB o x^J-'^v^x'? I-Tlt'&f 
S C i «!: Tf^fig L /cS^^;U^P< X -< 7^* ^" v*x ^ 

[003 1] J.XT(tccD^^-7;U5^;><x-i'7'^ x-? 

[0 0 3 2] C05g8S^««T-<O:*:*?tis AtBOn 
a t u r a 1 (7)laTa*n*3b\ C<^ili-&t* 10 + 1 
5T'2 5T'^?.„ 

[0 0 3 3] ^zLz-mTiSt. ^tw^1sts^ii^ qst^ 

^AtBT'ftSt-'SCltlC^SJb^ CWfefiSttAtB 
(Ds h r i n k/\^^— ^CDJttC J:oT@3»'rSo 

AODs h r i n kfi 1 OT'fefv BCDs h r i n 
k J* 2 «OT. @ 7 (^TST'u^ftf 
J:3tCAa)IHI*ai*1 > Ba)iHg*^»*5<!:^5. 
[0 0 3 4] 1'm^l.(r>}'-T'^T-^-y'J:L^V^^Wti 
lcJligElc«^LrcJS^<D«^^;Uf^^x-i' 7';i-^ v'x 

©shrin k/\°5;^-^(DltlcJ:^T^*B^a^ik«»J 

[0 0 3 5] e:tD^^-7;L/^p<x-r7':*-yv''x'9 hC5 
Wft-Ji^lix AfcJ:lfB<7)<*i;fi<7)»lU^;£«. Ji«> 
Si^^ttiiKlc s t r e t c h/\'^p<— JiCDJtlCjJo 

[0 0 3 6] 2m±<^)^x-<7':i-:?'v'x'? h:&^Fa« 

h^WnS^t. =&*8fi8S^<^)P<X'f7':i-:'"v'x'? h 
©s t r e t c h/\°^/— S'iDltlC.fcoT^^t^S^-Jt 

[0 0 3 7] J.X±)!)\ StartEndB o x^^Vx 
^ h»c«l:«B#F^e<jlcai|?SH:yx-f 7'*:?'i>x<7 
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[0 0 3 81 (b) mrsmicm.¥nz^=r^T^y''Jj:<7 

^•r«1t-&*:?v*"x^ T o p B o t t omBo X 
[0 03 9] TopBottomBo xt*. ItSO^x 

[0040] T o p B o t t omB o x^:?'v'x'? 

$e«s a^s^(7)Sp<7=^-<7':i-:?'i>'x^7 hG)#ai®itic 

■To 

[0 0 4 1 ] l2l8T1*s (natural, shrin 
k, stretch) a)§#$S/\°7;><-^'3b^ (1 0. 
2, 7) ©:t:?"v''x^hA<!:. (15, 10, 1 ) 
:? v*"x^ hBCD2-p©;><7='-i'7':t:/v*'x<7 h:&^Fa«lC 

;t:/v*x^ h^T opBottomBo if.-^'J-J:^^ h 

[0 0 4 2] Ji(TiciicDa^^;i/^/x-<7':t:?v'x';:7 

[0 04 3] ^-fx M^JlC^^-r^S^x-f 7':t:?v'x 

10-2 <r> 8 ft^Sx 1 0 + 7(01 7*T'0FaT-fe 
So $fcs B<3D:*:*#<!:LTlXy^#S<S^*. ^Wi^SSA" 
^P<— Sf(Dfii*^6tt»r«tx 15-10(D5*^6. 1 
5 + 1 05 1 6 « T(DFaT-^*c 

[0 0 4 4] -.!K\^'^T(r:>J^^^T^'f'^:!^^^(^-^'^^ 
<!:LTlXy^iSfiiro«ii©H55R«>«o EI8(7)Ji^, 8 
1 6$T'T'fe5o 

[0 0 4 5] ii(DJ^>it5HlJ::^T<D«^*tv«;><7=^'f T' 
Tl-y-Ji.^ hOn a t u r a I /\°^^— ^Offlft'^iR^ o 
Tt^'nt*', cn6<^)n a t u r a I <Di&(0^tt'TSt±<D 
tOD^^-^^yl/f'^x-f T'^d-T/v^x'? KDg^ft (na 
t u r a I ) t'^Mt^o 

[0 0 4 6] Cfl)Ji'&a)s h r i n k<Dlit*^ rj^iltS 
HCO±W - ^'&^/L/^^x-f7'7t-'?'v'x^ hCDg^ 
:lj ICJ^^x s t r e t c h05fiil*x r^-&-7;U^p<x 
'('7'*-?>>'x'? hcDS^S - tt>lt5H<0TWJ 

[0 04 7] E18C0Jia^*^ A*5«fctf BCDn a t u r a 
MiS'l Oil 5T»*a)T\ ttlcJtiltBffllciRS-p 



Tl^So L/cati^oTs CCDg8SS»AtB<Dn a t u r 

a \ mo'^tS'^mi^m^fR-or ^ strt^o 

[0 0 4 8] $fc. C<0stretcht*16-15<D 
1, s h r i n klil 5-8©7<i:%5<, 

[0 0 4 9] «a®Hic^T©«^*n5p<x'f7'7t-:?' 

h©n a t u r a I (Dii*^JR«6^:t"'«^»*. 

[0 0 5 0] CODJi^ros h r i n k<Oimt. rj^iliS 
H0±^ (=:^^^/l/f^;»<x-<7';if-:/v^x'>hcDe^ 
m) - ^^iSESHOTWJ iC^Us s t r e t c hi* 

[005 1] J-X±*\ TopBot tomBo x^^'J 
x^7 hlcJ:S^F^WlcM9JlCP<X'f Z:t:?v>'x'? 

[0052] EIHC, StartEndBoxtTop 
Bo t t omBo x^fflfT^SCtO/x-f 7':t:^-V'X'? 
h«ffl*'^to-y-T^^V/U9^;»«X'f 7':t:? v>x^ h^^ff 

[00 5 3] ^W*:*:*<»ltT«^;'<X'f7':t:?v>x 

So 

[0 0 5 4] 1fi-&;><xV7':d-7'v>'x'? hg3l^gp«^l:: 
TopBot tomBox ^yjji ^ hT-^-&*tl/c^ 

;i/^yx'f7';d-:/v»'x-? hro#aiSttfil*-^^U ¥- 
:?'>>■ 1^:7 hSaiSSPttilCS tartEndBox:t 

#ffijittffl«'3iS-rSo 

[0 0 5 5] 1 T^A^*tl/i:#P<X'f7':d-:?'i>'x'57 
#*SJltt»*s 3 (D=l h1$S3i5lCjli6tlSo 
=l>/1^-^> hfiiaSPtiS tartEndBo x.ytZf'J 
h(C«i:-pTB5F^MH:ai^lC«^S-n/i:yx-i'7':t:?' 

[0 0 5 6] S t a r t E n d Bo xTHSn/cfliJSEC- 

agPlCjJienSo 4TI*s TopBo t t omBox^ 
^i>'x h (c J: ^T^F^Wlca£5<Jti:ffi^i*-nfcP< X 1- 7 

So 

[0 0 5 7] CdT's :^**(Dtt®ESH6^^i6tl^ltn 

\f. 5(7)»^^4S!^?T*#aisi4fii<D<«iE«ffi-ro 
iE^fToJi^i* 5 - 1 <D#!ass14^^IESl^^c^T#s s^*^ 
»«^•nn^^:5-20x^-^tflssl5'^^T<o 4T'#eti 

/c«SSi!Hl*6<Dg5g:t^v*'x'> hSfJBraJ'SjTI* +1 
So CdTI*. P#FaWlcll£5iJlcffl^*+vSg^p<7=''f 
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■^fJ:L^ VOiVi a t u r a I / P< — 51 *i««jimcOK 
HlciRS^TU^W* rsn a t u r a I 0-5'6S:^CDt> 

^(r>msi<T)m.±m ^«^:!i-:?'v'i^ hStLTSdsr 

10 0 5 81 U±)b'!«^y7=^'f hgj^Sas 

[0 0 5 91 Start EndBox;*-:?" 

v*i^7 h^To p Bo t t omB o V^m 

<i:LT«7)fffii®'t4ffl/\°^^— ? (natural, s 
hrink, stretch) So 
[0 0 6 01 ^—t-?'J:L^ h-M)*SgPT»x «^^7^ 

>h (S t a r t EndBox7)-:?^S>x-^hT'^^*n 

ic»is«i« (8-1) ^tc\tm\-mk (8-2) tcs* 

[00611 8-1 CDffJii^«*J«fcO= 8-2 
T«x StartEndBo x?|-:?'-:/x'? KD^^JiiJ 

[0 0 6 21 J.X±T-^aV;Uf^yx-<7'?|-:?'v>x'5' hft 

So 

[0 0 6 31 Stlcff^JSF^<D^'&^;Uf'^xf7' 
X h (c*t LT^«iE-¥'^M*^rax.sJi^^i^ 5 - 1 

#nS, 
[00641 

^^yu^^x-f 7';t:^v>'x-? i-«lfi2:&^Ttt. WTWJa 

[0 0 6 51 2|s:|8^0ffi^^;U^yx-<7'«lfiB:^^T' 

-fT'Tj-^^v^x^' hiis yx-('7'?f-:?'v'"x^ hFa«-«s^-r 

Sa5» (TemporalGlue) lC)^Li*SSS14« 
^AU ^7=^-fZ:t3^v'x'? h<D^*^SSI^i^P<7=^^'7':l- 
[0 0 6 61 iie^T> <fffl«*'««^^/U^P<x-i'7'=J-:^" 
v^i^ h^«^-rsPSiic. ^x-r7'7|-:?'i>"x^^ hF^©[5| 



[0 0 6 7] ^rc, 2|s:fl^Wm^V;U^/x-l'7':i-:?"5>" 
X-? St3R<^;?ie©J:^lcB$F^fi6±tcy 

«glClfi»(3D^x-<7':t:?v*'x'> h^«^LT— OC^/x 
-{Yz^y-JzL^P VKt^rzisbOS tartEndSo 
X . fl$Fa6*Ilca£5'Jl::ffia<D^ 7:^y=J:sL<? h 
LT— -PflDy^-f 7':?|-:?"v*'x'57 Kc-rSfc»(OTo p B 
o t t omBo x^<tf6Da-&?t- X •J' h^ffil-'Ts SP 

[0 0 6 81 ^JtoT. «^?yU^^x''f7':t3^v'x'7 h 

^f^fiRf set RitBi TL^ -So 
[0 0 6 91 Z.<orcli>. 7:<D®Si^—-Z><0}t9-(7^-j^ 

iES*SM-r?>l^lCl*> — :>0;><X'r7'7t-:?'v*x^' 

[0 0 7 01 S/c. Slf^#A!«Bt(«:f^fiKS53^ro^^v^l/f^ 

sJ^STSorc. Bljl-rS^T(7);t:?'-:^x"9 h-CD^tlEit'* 
[0 07 11 «<J;5H*EI9 (a) SS->-V<^^7=-< 
•&t?1*T. ^■CD>'-V<!:BGM*i'i|lBtlcSltoS<ll^«^ 

[0 0 7 21 dOJi'&t*, 09 (a) a)^v^c•/->^ 

CD/x-<7':t:?v'x-i7 hlcWLT. BGMtT empo 
r a I G I u e?|-:>'i>'x'> K^StartEndBox 

T-^^L/cptx-f T'^^v'x^^ h*ffl*Lx ^ne^H 
9 (b) ©Jc^lCTo p B o t t omB o xTS^fS 

lix Tempo r a I G I u e<^gPiJ««#i!e-r«C<i:li: 

[0 0 7 31 $fcx 1219 (c) 03,»:^lc->-V1 

y'Jji^ t>«T emp o r a I G I u e ©gPiJfi^ffiiSr 

^c^73^:i:3b'«T'*. Sf3l5^iC<DJ:^lc. S^/ctc«^^;U 

icia^iij^-rSC <!:*^iiS3KS. 
[^SaOfSW^m^l 

[01] ^^^<r>m.'^-^)\^=f-}^9-(T:i-y'j:!i<P ymm. 

[02] $^3R(0^'-<A^-f>tx/U^ffll^T^^^/U^ 
^x-f 7'7f-:^ v'x'> h^«^rS«?^u^L/ct.<DT-^ 

So 



(7) 
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[03] 'iStikO^^L^^^y^y'MZ^^mm'^T.'rh, 

[05] :$immo-mmm^mi^T. 7=^-1- 

[06] *SIWtcJ:«^7=^'f 7':i-:?'->i^ h^ffffiStt 
[07] *^R^»c.i:*^x-f 7'7|-:^v'i'j7 h<D#JSB14 

[08] *Sl^lcJ:l.p<xf T'Tj-T/v'x'? hOffiSBIS 

[09] *^^<Djams^-r0T'»»j. ^aw/x-i-T' 



[^^osi^] 

1 

2 
3 
4 
5 



5?«i*:^v> x-j7 KOjf A. 
5 - 1 #JSJi14ffl<gIESI5 

5-2 x^-iasgp 

8 - 1 w^mm 
8-2 ag/mts 

9 *-;t^ v' x^ 

1 0 ^7t-:? v' x^ hg-*^ 



II] 



X 



5-1 



E5H 



r 5-2 



10 



TZ* 3 



1 „ 




f 8 -1 ^ 8 - 
i i 
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(54) COMPOSITE MULTI-MEDIA OBJECT CONSTITUTING METHOD 

(57)Abstract: 

PURPOSE: To easily execute the generation/correction by leading in an 
expansion/ contraction attribute related to time with respect to media data and a 
coupling part between the media dataand constituting a composite multi-media 
object by the expansion/contraction attribute. 

CONSTITUTION: An expansion/contraction attribute of a media object is sent to a 
component inspecting part 3 and a range of size of the media object connected 
and compounded timewise is inspectedand alsosent to a common length inspecting 
part 4 and a common range is calculated. Unless the common range of size is 
obtained thereinit goes to a warning generating part 5 and a correction of an 
expansion/contraction attribute value is urged. In the case of a correctionit goes 
to an expansion/contraction attribute correcting part 5-1 and if a warning is 
disregardedit goes to an error processing part 5-2. The obtained common rangs is 
delivered to an optimal object length deciding part Band when a natural parameter 
of an element media object compounded in parallel timewise is held within a range 
of common lengthits maximum becomes composite object lengthand in other 
casethe maximum value of common length becomes composite length. 



CLAIMS 



[Claim(s)] 

[Claim 1]In a method of constituting a compound multimedia object from on a 
computercombining media data with a dimension about timesuch as video and an 
audiotwo or moreA compound multimedia object constitution method introducing 
an elastic attribute about time and constituting a compound multimedia object with 
said elastic attribute to a connecting part between said media data and said media 
data. 

[Claim 2]Said elastic attribute has natural length and the amount of the maximum 



shrinkage of a media objectWhen compounding two or more media objects with 
continuation in timein creating a compound multimedia object shorter than the 
sum of natural length of two or more of said media objectsThe multimedia object 
constitution method according to claim 1 calculating a ratio of the amount of the 
maximum shrinkage which said two or more media objects havereducing each 
media object according to a ratio of this amount of the maximum shrinkageand 
creating said compound multimedia object. 

[Claim 3]Said elastic attribute has natural length and the maximum stretch amount 
of a media objectWhen compounding two or more media objects with continuation 
in timein creating a compound multimedia object longer than the sum of natural 
length of two or more of said media objectsThe multimedia object constitution 
method according to claim 1 calculating a ratio of the maximum stretch amount 
which said two or more media objects havelengthening each media object 
according to a ratio of this maximum stretch amountand creating said composite 
object. 

[Claim 4]Said elastic attribute has the amount of the natural length and maximum 
shrinkage and the maximum stretch amount of a media objectWhen compounding 
two or more media objects in parallel in timesearch for a common range which the 
length of two or more of said media objects can takeand if said natural length of 
two or more of all the media objects of said is said less than common rangeThe 
greatest natural length in said two or more of said media objects is made into 
natural length of said composite objectif a nether world of said common range is 
made into the amount of the maximum shrinkage of said composite objectan upper 
bound of said common range is made into the maximum stretch amount of said 
composite object and a thing besides said common range has said natural length in 
said two or more media objectsMake the maximum of said common range into 
natural length of said composite objectand a nether world of said common range is 
made into the amount of the maximum shrinkage of said composite objectThe 
compound multimedia object constitution method according to claim 1 setting the 
maximum stretch amount of said composite object to Oand constituting a 
compound multimedia object. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application]This invention relates to the method with which an operator 
realizes the multimedia authoring tool which supports the work which creates a 
compound multimedia object on a computerfor example. 
[0002] 

[Description of the Prior ArtjWhen creating one work (compound multimedia 
object) combining two or more media objects with the attribute about timesuch as 
video and an audioediting work is performed using the editing system called an 



authoring tool and an editor. An editing system defines the synchronous 
relationship between media objects by a certain formand supports the work which 
constitutes one compound multimedia object eventually. Media objects are each 
media data of videoan audioetc. used as the parts of a compound multimedia 
object. 

[0003]The conventional editing system defined one time-axisin order to arrange 
each media objectand the time-line model form of an operator putting a media 
object in order and going on the time-axis is taken. 

[0004]In this modelgenerallyas shown in drawing 2 (a) and drawing 2 (b)two or 
more tracks in alignment with a lateral time-axis are preparedand one media 
object is assigned on each track. A media object is displayed as a rectangle of the 
size proportional to the time lengthand an operator is operation by a 
mouseetc.The synchronization between media objects will be specified by 
arranging each media object on this tracktaking into consideration the relation 
between a time-axis and a rectangular size. 

[0005]Each track of arrangement of the media object by this time-line model form 
is completely independentand the synchronous relationship of two or more media 
objects is described by by being arranged at the same coordinates on a time-axis. 
[0006] 

[Problem(s) to be Solved by the Invention]Howeverin the conventional methoda 
style and when correctingthe following problems existed the compound multimedia 
object. 

[0007](1) Since there is no elasticity in an inflexible media objectwhen combining 
two media objectsthere is no mechanism in which the length of another side 
corresponds dynamically according to one length. 

[0008]For examplethe case where a compound multimedia object which the scene 
and BGM finish simultaneously is created combining a certain scene and BGM is 
considered. That isit will start and BGM will use so that it may become the same 
as the length of the scene 1. At this timewith the editing system by the 
conventional time-line modelas shown in drawing 3 (aXhe scene 1 is assigned to 
the track 1BGM is assigned to the track 2and an operator arranges so that the 
last of the scene 1 and an operator's last of BGM may correspond on the basis of 
a time-axis. 

[0009]When the length of the scene 1 becomes longer than that whose intention it 
had first to this compound multimedia objectin the conventional method. The 
length of BGM remains as it is like drawing 3 (b)and in order to coincide the last of 
the scene land the last of BGMBGM must be changed for a long time by an 
operator s own hand like drawing 3 (c). 

[0010](2) Correction in the conventional synchronous specification by difficult 
time-line form. Since it is not specified whether it is the point that which position 
between two or more media objects should synchronize trulylt is unknown which 
object when correcting a part of created compound multimedia object alreadyit is 
related and must be correctedif it does not manage on an operator's 
responsibilityit learnsand it is **. 



[001 1]For examplethe compound multimedia object shown in drawing 4 (a)Video2 
sometimes starts with the start of Video4 and Videol and Audiol are combined 
with the compound media object (let this be the compound media object 1) 
constructed one that Video3 is completed simultaneously with the end of Video4. 
As the wholeVideol and Audiol begin simultaneously and the compound media 
object 1 begins after a while after completing Videol. And it is an example of the 
compound multimedia object that the end of the compound media object 1 and the 
end of Audiol are in agreement. 

[0012]The case where a part of this compound multimedia object is corrected is 
considered. For examplesuppose that Audiol is changed into a long thing. It is 
necessary to be change of a starting position also about Video2 and Video3 which 
had to change the starting position of Video4 which suited the synchronous 
relationship of Audiol with change of sizeand suited the synchronous relationship 
of Video4 further. All of these change must be performed in the responsibility of 
the operator who editsand the work of creation/correction is inefficient-like about 
a complicated compound multimedia object. 

[0013](3) A part or all of a compound multimedia object with low reusability as 
parts that has already been created cannot be used as parts when creating other 
compound multimedia objects. This is because synchronous specification of the 
compound multimedia object is made by the absolute coordinates on a time-axisso 
the absolute coordinate cannot be applied as it is when including in other 
compound multimedia objects. Thereforealso when creating the compound 
multimedia object which changed only the part from a certain compound 
multimedia objectall must be remade from the beginning. 

[0014]For example drawing 5 (a) uses an image and a sound effect as the parts 
constructed one (it is considered as the parts Dand is the example which created 
the compound multimedia object combining the parts (it is considered as the parts 
2) to construct one about English narration and a Japanese title. When changing 
the portion of this part 2 to the parts 3 which consist of Japanese narration and 
an English title and creating a new compound multimedia objectif constituted from 
a conventional methodthe whole also including the portion of the parts 1 must be 
remade and reuse of the parts 1 is difficult. 

[0015]Thusthere was a problem which should be solved in the conventional 
compound multimedia object composition method. 

[0016]The purpose of this invention solves the above-mentioned problemand can 
make creation/correction easily for a userRealize the compound multimedia object 
composition method which can reuse the compound multimedia object furthermore 
created as parts at the time of creating other compound multimedia objectsand. It 
is realizing the playback system for reproducing the compound multimedia object 
created based on the compound multimedia object composition method in this 
invention. 
[0017] 

[Means for Solving the Problemjin this inventionthe above-mentioned problem is 
solved by using the following techniques. 



[0018]In a way this application 1st invention constitutes a compound multimedia 
object from on a computer combining media data with a dimension about timesuch 
as video and an audiotwo or moreTo a connecting part between said media data 
and said media dataan elastic attribute about time is introduced and said elastic 
attribute constitutes a compound multimedia object. 

[0019]In this application 2nd inventionsaid elastic attribute has natural length and 
the amount of the maximum shrinkage of a media objectWhen compounding two or 
more media objects with continuation in timein creating a compound multimedia 
object shorter than the sum of natural length of two or more of said media 
objectsA ratio of the amount of the maximum shrinkage which said two or more 
media objects have is calculatedeach media object is reduced according to a ratio 
of this amount of the maximum shrinkageand said compound multimedia object is 
created. 

[0020]In this application 3rd inventionsaid elastic attribute has natural length and 
the maximum stretch amount of a media objectWhen compounding two or more 
media objects with continuation in timein creating a compound multimedia object 
longer than the sum of natural length of two or more of said media objectsA ratio 
of the maximum stretch amount which said two or more media objects have is 
calculatedeach media object is lengthened according to a ratio of this maximum 
stretch amountand said composite object is created. 

[0021]Said elastic attribute this application 4th invention Natural length of a media 
objectWhen it has the amount of the maximum shrinkageand the maximum stretch 
amount and two or more media objects are compounded in parallel in timeSearch 
for a common range which the length of two or more of said media objects can 
takeand if said natural length of two or more of all the media objects of said is said 
less than common rangeThe greatest natural length in said two or more of said 
media objects is made into natural length of said composite objectif a nether world 
of said common range is made into the amount of the maximum shrinkage of said 
composite objectan upper bound of said common range is made into the maximum 
stretch amount of said composite object and a thing besides said common range 
has said natural length in said two or more media objectsThe maximum of said 
common range is made into natural length of said composite objecta nether world 
of said common range is made into the amount of the maximum shrinkage of said 
composite objectthe maximum stretch amount of said composite object is set to 
Oand a compound multimedia object is constituted. 
[0022] 

[Function]The basic principle of the compound multimedia object composition 
method of this inventionBy introducing an elastic attribute to all the media objects 
used as the parts of a compound multimedia objectin compound multimedia object 
activitythe complicatedness of the synchronization between each media objects is 
reducedand the reusability as parts of the created compound multimedia object is 
improvedand it aims to reduce the complicatedness of the operation in the case of 
correction. 
[0023] 



[Example]The method of introducing the elastic attribute of the introduction media 
object is explained. 

[0024]As shown in drawing 6 each media object comprises three parametersnatural 
(natural length)shrink (the amount of the maximum shrinkage)and "stretch (the 
maximum stretch amount)." Thereforewhen the external force about elasticity is 
not applied to a certain media objectit becomes a size expressed with a natural 
parameter (when it is not lengthened or does not contract). Since the maximum 
stretch amount of a media object is expressed with a stretch parameterthe size of 
the media object at the time of being lengthened most turns into a size expressed 
with natural+stretch. 

[0025]When shrunkensince the amount of the maximum shrinkage of a media 
object is expressed with a shrink parameterthe size of the media object at the 
time of contracting most turns into a size expressed with natural - shrink similarly. 
[0026]Also about the connecting part between media objects when a compound 
multimedia object is constitutedthis is treated noting that it is one of the special 
examples of a media objectand the same elastic attribute as the above-mentioned 
media object is introduced. Herethe connecting part between this media object is 
called TemporalGlue. 

[0027]By introducing TemporalGluethe connecting part between the mere media 
objects which presupposed that it is blank in time and were treated can treat by 
the conventional method as one media object which can be expanded and 
contracted free. 

[0028]The composite object which performs operation for creating a compound 
multimedia object is explained combining the media object which was able to give 
the elastic attribute as mentioned above. 

[0029](a) Composite object which compounds a media object with continuation in 
time in composite object this invention which compounds a media object with 
continuation in time It is referred to as StartEndBox. StartEndBox is a composite 
object which performs operation which compounds two or more media objects with 
continuation in timeand creates a new compound multimedia object. Elasticity of 
the compound multimedia object created using the StartEndBox object is 
determined by the elastic attribute of each media object of a compound element. 
The amount of elasticity in the case of compounding two or more media objects 
with continuation in time is shown in drawing 7 . 

[0030]The case where each elastic parameter in drawing 7 (naturalshrinkstretch) 
compounds twothe media object A of (1027) and the media object B of 
(15101)with continuation in time is shown. The compound multimedia object 
created by compounding these two media objects by a StartEndBox object is set 
to C. 

[0031]The numerical orientation method of the value of the elastic attribute of this 
compound multimedia object C is explained below. 

[0032]The size in the natural state of C is in this casealthough expressed with the 
sum of natural of A and B. It is 25 in 10+15. 

[0033]When contracting the whole from 25 to 19 herethe whole amount of 



shrinkage is 6. This amount of shrinkage Although 6 will be paid by A and Bthis 
amount of burdens is distributed by the ratio of the shrink parameter of A and B. 
That issince shrink of A is 10 and shrink of B is 2as for the amount of shrinkage of 
A1 and the amount of shrinkage of B are set to 5 so that it may be expressed with 
the formula shown in the lower berth of drawing 7 . 

[0034]Also when contracting the compound multimedia object at the time of 
compounding two or more media objects with continuation in timethe total amount 
of shrinkage is distributed in proportion to the ratio of the shrink parameter of the 
media object of each component. 

[0035]Are contrary to the time of contracting the calculating method of the 
stretch amount of A and Bwhen lengthening this compound multimedia object C. 
The ratio of a stretch parameter distributes. 

[0036]Also when lengthening the compound multimedia object at the time of 
compounding two or more media objects with continuation in timethe amount of 
total elongation is distributed in proportion to the ratio of the stretch parameter of 
the media object of each component. 

[0037]It is a deciding method of the elastic attribute of a compound multimedia 
object at the time of [ which the above depends on a StartEndBox object ] 
compounding a media object with continuation in time. 

[00381(b) Composite object which compounds a media object in parallel in time in 
composite object this invention which compounds a media object in parallel in time 
It is referred to as TopBottomBox. 

[0039]TopBottomBox is a composite object which compounds two or more media 
objects in parallel in timeand creates a new compound multimedia object. 
[0040] Elasticity of the compound multimedia object created using the 
TopBottomBox object is determined by the elastic attribute of each media object 
of a compound element. The amount of elasticity in the case of compounding two 
or more media objects in parallel in time is shown in drawing 8 . 
r0041] Drawing 8 shows the case where each elastic parameter of 
(naturalshrinkstretch) compounds two media objectsthe object A of (1027)and the 
object B of (15101)in parallel in time. The compound multimedia object created by 
compounding these two media objects by a TopBottomBox object is set to C. 
[0042]The numerical orientation method of the value of the elastic attribute of this 
compound multimedia object C is explained below. 

[0043]Firstthe size which the media object can take from the elastic attribute of 
each media object compounded in parallel is calculated. The value which can be 
taken as a size of A in the case of drawing 8 is 10-2when it calculates from the 
value of the elastic parameter. It is before 17 of 8 to 10+7. When the value which 
can be taken as a size of B is calculated from the value of the elastic parameterit 
is before 1 6 of 5 to 1 5+1 of 1 5-1 0. 

[0044]Nextthe common range of the value which can be taken as a size of all the 
media objects is searched for. In the case of drawing 8 it is from eight to 1 6. 
[0045]If the value of the natural parameter of all the media objects compounded is 
settled in this common rangethe greatest thing in the value of these natural(s) will 



be determined as the natural length (natural) of a compound multimedia object. 
[0046]The value of shrink in this case becomes "Upper bound of the common 
range - the natural length of a compound multimedia object"and the value of 
stretch serves as "a nether world of the natural length -common range of a 
compound multimedia object." 

[0047]In the case of drawing S since the natural values of A and B are 10 and 15it 
has fitted in both common ranges.Thereforethe natural length of C takes the 
maximum out of the natural value of A and Band is set to 1 5. 
[0048]stretch of C is set to 1 of 16-15and shrink is set to 7 of 15-8. 
[0049]When the value of natural of all the media objects compounded is not 
settled in the common rangelet the upper bound of the common range be the 
natural length of a compound multimedia object. 

[0050]the value of shrink in this case becomes "a nether world of the upper bound 
(natural length of = compound multimedia object) - common range of the common 
range" — stretch It is set to 0. 

[0051 ]It is a deciding method of the elastic attribute of a compound multimedia 
object at the time of [ which the above depends on a TopBottomBox object ] 
compounding a media object in parallel in time. 

[0052]When StartEndBox and TopBottomBox are used for drawing 1 and a 
compound multimedia object is created combining two or more media objectsAnd 
the deciding method of the elastic attribute value of a compound multimedia 
object at the time of correcting a created compound multimedia object is shown. 
[0053]The whole is roughly divided and comprises a compound media object length 
deciding part and a single object length deciding part. 

[0054]A compound media object length deciding part determines the elastic 
attribute value of the multimedia object mainly compounded by the TopBottomBox 
objectA single object length deciding part determines the elastic attribute value of 
the multimedia object mainly compounded by the StartEndBox object. 
[0055]The elastic attribute value of each media object inputted by 1 is sent to the 
component inspection section of 3. The range in which the size can take a 
component inspection section about the media object compounded in time with 
continuation by the StartEndBox object is inspected. 

[0056]The range which the size which was surrounded by StartEndBoxand which 
was calculated for every field can take is sent to the common length inspection 
section of 4. In 4the common range of the range which the size of the media 
object of the component can take is calculated about the media object 
compounded in time by the TopBottomBox object in parallel. 
[0057] Hereif the common range of a size is not acquiredit goes to the alarm 
generating part of 5and correction of an elastic attribute value is urged. If it goes 
to the elastic attribute corrected part of 5-1 and warning is disregarded when 
correctingit will go to the error-handling part of 5-2. The common range acquired 
by 4 is passed to the optimal object length judgment part of 6. If the natural 
parameter of the element media object compounded in parallel in time is here 
settled in the range of common lengththe greatest thing among each naturalwill be 



made into composite object lengthin the case of othersthe maximum of the range 
of common lengthis determined as composite object length. 

[0058]The above is the composition of a compound media object length deciding 
part. 

[0059]So farthe elastic attribute value parameter (naturalshrinkstretch) as the 
whole compound multimedia object constituted using the StartEndBox object and 
the TopBottomBox object is determined. 

[0060]At a single object length deciding partit is a compound media object length 
deciding part. It is based on elastic ******** of the compound multimedia object 
determined by AThe elastic attribute value of the media object in all the 
components (multimedia object compounded by the StartEndBox object) contained 
in a compound media object is determined. An elastic mechanism (8-1) or a 
reduction mechanism (8-2) is passed for every componentand the length of a 
single object is determined eventually. 

[0061 ]In the elastic mechanism of 8-1 and the reduction mechanism of 8-2the 
quantity which is proportional to an elastic attribute value based on the compound 
rule of a StartEndBox object is expanded and contracted. 
[0062]The time size of all the components included in a compound multimedia 
object above is determined. 

[0063]When adding correction and change to a created compound multimedia 
object alreadyafter correction is added by the elastic attribute value corrected 
part of 5-1 the result is sent to the component inspection section of Sand all the 
media object length is determined in the same procedure as the case of a new 
input henceforth. 
[0064] 

[Effect of the InventionjAs explained to details abovein the compound multimedia 
object composition method of this inventionit has the following effects as a 
feature. 

[0065]AII the media objects which serve as a component of compound multimedia 
in the compound multimedia composition method of this inventionAn elastic 
attribute is introduced to the portion (TemporalGlue) which combines between 
media objectsand it has a form which embeds the elastic mechanism of a media 
object at the media object itself 

[0066]Thereforewhen a user constitutes a compound multimedia objectabout the 
specification of a synchronization between media objects. Only operation which 
compounds a media object with continuation in timeand operation compounded in 
parallel in time are only performedand since the fine adjustment operation between 
media objects performs [ the object itself ] automaticallyan operator s work 
reduces it substantially. 

[0067]In the compound multimedia object composition method of this invention. A 
media object is not arranged on a time-axis like the conventional 
methodStartEndBox for compounding two or more media objects with continuation 
in timeand making it one media objectit is a method which assembles parts and 
goes using composite objectssuch as TopBottomBox for compounding two or 



more media objects in parallel in timeand making it one media object. 
[0068]Thereforeit is possible to create parts with a mass of meaning which forms 
a certain portion of a compound multimedia object. 

[0069]For this reasonwhen it is possible to treat this part as one media objectand 
to constitute a still more complicated compound multimedia object and 
arrangement of that part is changed. The complicatedness of the work of 
correcting all parts related like [ it is possible to operate it as if it corrected 
arrangement of one media objectand ] the conventional method is mitigable. 
[0070]Also when an operator corrects a created compound multimedia object 
alreadythe operator can cope with it only by change of an elastic attribute 
valueand the work of a required correction of all the related objects becomes 
unnecessary by the conventional method. 

[0071]For example drawing 9 (a) is an example which creates a compound 
multimedia object which the scene and BGM finish simultaneously combining the 
media object of a certain sceneand the media object of BGM. 

[0072]In this casethe media object of the scene 1 is received like drawing 9 (a)The 
media object which compounded BGM and a TemporalGlue object by StartEndBox 
is preparedand they can be created by compounding by TopBottomBox like 
drawing 9 (b). At this timethe starting position of BGM is automatically arranged in 
a suitable positionwhen the portion of TemporalGlue expands and contracts. 
[0073]Even if it changes the scene 1 into that from which a size is different like 
drawing 9 (c)When the portion of TemporalGlue expands and contractsthis 
compound multimedia objectCorrection which has saved the whole structure can 
be made easily and a man day can be substantially reduced like a conventional 
system compared with the work which newly constitutes a compound multimedia 
object. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] How to determine the elastic attribute of the media object using the 
compound multimedia object composition method of this invention is shown. 
[Drawing 2] Signs that a compound multimedia object is constituted using the 
conventional time-line model are shown. 

[Drawing 3] The example of 1 edit in the editing system by the conventional time- 
line model is shown. 

[Drawing 4] The example of 1 edit in the editing system by the conventional time- 
line model is shown. 

[Drawing 5] Signs that a compound multimedia object is constituted are shown 
using one example of this invention. 

[Drawing 6] The elastic attribute of the media object by this invention is shown. 
[Drawing 7] It is a figure showing the numerical orientation method of the elastic 
attribute value of the compound multimedia object which compounds with 



continuation in time and can do two media objects using the elastic attribute of 
the media object by this invention. 

[Drawing 8] It is a figure showing the numerical orientation method of the elastic 
attribute value of the compound multimedia object which compounds in parallel in 
time and can do two media objects using the elastic attribute of the media object 
by this invention. 

[Drawing 9] It is a figure showing the effect of this inventionand signs that a 
compound multimedia object is created combining two or more media are shown. 
[Description of Notations] 

1 The input of an elastic parameter 

2 Insertion of a new objectdeletion 

3 Component inspection section 

4 Common length inspection section 

5 Alarm generating part 

5-1 Elastic attribute value corrected part 
5-2 Error-handling part 

6 Optimal object length judgment part 

7 Composite object length determination 
8-1 Stretcher style 

8-2 Reduction mechanism 

9 Single object length determination 

10 All the object length determination 



